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Executive  Summary 


The  1996  version  of  the  Human  Factors  Design  Guide  (HFDG)  consolidated  multiple 
sources  of  human  factors  guidance  and  overcame  limitations  associated  with  using 
military  standards  and  guidelines.  Upon  publication,  the  HFDG  quickly  became  a  key 
reference  tool  for  the  application  of  human  factors  policy  to  acquisitions  and  the 
development  of  new  systems  and  equipment. 

Researchers  at  the  Federal  Aviation  Administration  (FAA)  William  J.  Hughes  Technical 
Center  have  revised  the  original  Chapter  5  -  Automation  of  the  HFDG.  The  revised 
version  is  an  updated  and  expanded  set  of  automation  guidelines  to  meet  the  needs  of 
FAA  missions  and  systems.  Although  the  original  HFDG  was  primarily  focused  on  FAA 
ground  systems  and  equipment  such  as  those  managed  and  maintained  by  Airway 
Facilities,  the  researchers  expanded  the  coverage  beyond  maintenance  issues. 

Researchers  divided  the  revision  process  into  several  phases  including  the  identification 
of  source  material,  systematic  evaluation  of  literature,  reorganization  of  topic  areas,  and 
expert  review.  The  revised  chapter  is  limited  in  scope  to  human  factors  guidance  related 
to  automation.  The  complete  set  of  new  guidelines  is  contained  in  the  appendix.  The 
revision  resulted  in  several  changes,  as  follows. 

a.  The  search  for  current  information  pertaining  to  automation  resulted  in  the 
addition  of  102  new  sources. 

b.  The  review  of  these  source  documents  resulted  in  the  addition  of  over  100  new 
guidelines  that  researchers  incorporated  into  the  revised  document. 

c.  The  reorganization  of  the  revised  chapter  involved  an  extensive  regrouping  of 
information  as  well  as  the  removal  of  redundant  guidelines. 
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1.  Introduction 


The  modernization  of  the  National  Airspace  System  (NAS)  includes  an  increasing  reliance  on 
automation,  particularly  computer-based  automation.  Automation  in  the  modem  NAS  is  being 
used  in  more  places  and  in  more  ways  than  previously  considered,  due  largely  to  advances  in 
computer  technology  along  with  decreases  in  computing  costs.  Automation  has  the  potential  to 
reduce  workload,  reduce  errors,  and  increase  efficiency.  Improperly  implemented  or  designed 
automation,  however,  can  have  the  opposite  effect,  increasing  workload,  increasing  errors,  and 
decreasing  efficiency.  A  current  set  of  guidelines  to  use  as  a  reference  can  benefit  this  process. 

The  original  Chapter  5  on  maintenance  automation  in  the  Human  Factors  Design  Guide  (HFDG) 
(FAA,  1996)  contained  information  that  was  dated.  A  more  modem  set  of  guidelines  was 
needed  with  an  increased  scope  beyond  just  maintenance  issues.  This  document  summarizes  the 
development  of  an  updated  and  revised  set  of  automation  guidelines  for  the  HFDG.  Along  with 
the  introductory  material  briefly  describing  the  creation  process,  this  document  contains  the  new 
guidelines  as  an  appendix,  complete  with  a  table  of  contents,  sources,  and  an  index. 

1.1  Purpose 

The  purpose  of  this  document  is  to  provide  an  updated  and  expanded  set  of  automation 
guidelines  that  meets  the  needs  of  FAA  missions  and  systems.  Additional  goals  were  to  expand 
the  coverage  of  the  chapter  for  relevance  beyond  maintenance  systems  and  to  organize  the 
document  so  that  users  can  easily  locate  the  needed  information. 

1.2  Scope 

This  document  is  limited  in  scope  to  human  factors  guidance  related  to  automation.  Although 
alarms  and  alerts  are  tied  closely  to  automation,  guidelines  on  auditory  alarms  and  alerts  are 
being  consolidated  into  a  separate  chapter. 

1 .3  Shall  and  Should 


Each  guideline  specified  in  Appendix  A  is  identified  as  a  “shall”  or  “should”  statement.  A  solid, 
black  square  (■)  adjacent  to  the  guideline  identifies  the  "shall"  statements.  These  originate  from 
or  are  comparable  to  statements  from  authoritative  sources  such  as  those  associated  with  FAA 
orders,  standards,  and  military  specifications. 

Each  “should”  statement  is  identified  by  an  open,  white  square  (D).  These  represent  best 
practices  guidance  that  is  applicable  in  most  cases  but  may  involve  trade-offs  or  be  influenced  by 
context-specific  factors. 
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2.  Method 


Researchers  organized  the  revision  process  into  several  phases  including  the  review  of  original 
material,  identification  of  new  source  material,  systematic  evaluation  of  literature,  reorganization 
of  topic  areas,  and  expert  review.  During  this  entire  process,  the  research  team  tried  to  provide  a 
usable  reference  document.  This  meant  ensuring  that  guidelines  were  based  on  credible  sources 
and  were  stated  as  clearly  and  concisely  as  possibly,  with  minimal  redundancy. 

2. 1  Review  of  1996  Chapter  on  Maintenance  Automation 

In  the  first  phase  of  the  research  effort,  a  research  team  from  the  NAS  Human  Factors  Branch 
(ACT-530)  carefully  went  through  the  original  Chapter  5  on  maintenance  automation  from  the 
HFDG  (Federal  Aviation  Administration  (FAA),  1996).  This  original  version  contained  several 
guidelines  that  were  based  on  the  personal  experiences  of  previous  authors.  Guidelines  such  as 
these  that  could  not  be  verified  by  outside,  published  material  were  deleted.  Guidelines  that 
were  limited  in  scope  to  maintenance  automation  were  rewritten  where  appropriate  to  expand 
applicability  beyond  maintenance.  Other  guidelines  were  revised  as  necessary  to  make  them 
more  concise  or  more  understandable  by  allowing  only  one  “should”  or  “shall”  statement  per 
guideline.  Redundant  guidelines  were  deleted. 

2.2  New  Source  Material  Identification 

The  research  team  identified  and  obtained  automation  source  materials.  Most  of  the  information 
on  automation  came  from  disparate  journal  publications,  technical  reports,  and  books,  with  very 
little  in  the  form  of  guidelines.  The  researchers  evaluated  the  adequacy  of  the  source  material  for 
inclusion  in  the  revised  design  guide  chapter  and  rejected  documents  that  did  not  contain 
sufficiently  relevant  information. 

2.3  Literature  Evaluation 


The  research  team  compared  new  source  material  against  the  material  in  the  original  Chapter  5 
on  maintenance  automation  to  identify  areas  where  additional  information  was  needed.  When 
information  from  the  new  source  documents  warranted  the  creation  of  a  new  guideline,  relevant 
material  was  written  in  the  proper  guideline  format.  The  researchers  attached  the  applicable 
references  to  each  guideline  and  provided  a  list  of  sources  at  the  end  of  the  chapter.  Information 
on  automation  proved  much  more  difficult  to  make  into  guideline  format  than  computer  human 
interface  information,  thus,  the  guidelines  in  this  chapter  are  often  followed  by  extensive 
discussion  paragraphs  or  examples  to  further  clarify  the  guideline. 

2.4  Reorganization 

After  adding  all  of  the  new  material  in  guideline  format  and  revising  and  updating  the  existing 
guidelines,  researchers  reorganized  the  topics  and  guidelines  within  the  document  to  facilitate  the 
location  of  information.  They  grouped  related  topics  and  created  new  chapter  sections  from 
areas  that  had  been  expanded  due  to  new  information. 


2 


The  new  set  of  guidelines  are  divided  into  15  sections;  general  information,  design  and 
evaluation,  system  response  and  feedback,  interface,  user  acceptance  and  trust,  modes, 
monitoring,  fault  management,  false  alarms,  training,  function  allocation/levels  of  automation, 
information  automation,  adaptive  automation,  decision  aids,  and  control  automation. 

2.5  Expert  Review 

A  draft  of  the  revised  Automation  Chapter  5  was  circulated  among  a  select  group  of  human 
factors  professionals  for  their  review  and  comment.  An  automation  subject  matter  expert 
provided  further  review.  These  reviewers  provided  useful  feedback  on  the  chapter  organization 
and  the  effectiveness,  clarity,  and  relevance  of  the  guidelines.  The  automation  subject  matter 
expert  provided  additional  feedback  on  topic  completeness  and  provided  several  additional 
source  references.  The  research  team  addressed  all  of  the  comments,  revising  guidelines  and 
obtaining  additional  references  as  necessary. 

3.  Document  Overview 

The  revision  of  the  HFDG  (FAA,  1996)  Chapter  5  document  resulted  in  several  mentionable 
changes.  The  search  for  current  information  pertaining  to  automation  resulted  in  the  addition  of 
102  new  source  references.  The  review  of  these  source  documents  resulted  in  the  addition  of 
over  100  new  guidelines  that  were  incorporated  into  the  revised  document.  The  reorganization 
of  the  revised  document  involved  a  systematic  regrouping  of  information  as  well  as  the  removal 
of  redundant  or  unverifiable  guidelines. 

The  revised  Automation  Chapter  5,  contained  as  an  appendix  within  this  document,  provides  a 
comprehensive  set  of  usable  human  factors  guidelines.  As  with  any  set  of  guidelines,  those  that 
are  optimal  for  one  situation  may  not  be  suitable  for  another  situation  due  to  the  trade  offs 
involved  between  some  of  the  guidelines  and  context-specific  influences.  Consequently,  these 
guidelines  should  be  used  in  conjunction  with  the  advice  of  a  human  factors  expert. 

The  revised  Chapter  5,  as  a  part  of  the  HFDG  (FAA,  1996),  is  considered  a  living  document.  It 
will  be  updated  as  needed  to  keep  current  with  emerging  research,  technological  advances,  and 
user  feedback.  This  will  provide  the  most  current  human  factors  knowledge  in  a  usable  tool. 

In  creating  these  guidelines,  the  researchers  have  tried  to  develop  an  easy  to  use,  comprehensive 
reference  document.  Effort  was  made  to  reduce  redundancy  while  providing  clear, 
understandable  guidelines.  However,  no  document  is  without  room  for  improvement. 
Constructive  remarks  on  this  document  can  be  sent  to  the  authors  at  the  William  J.  Hughes 
Technical  Center,  Atlantic  City  International  Airport,  NJ  08405. 

The  update  of  HFDG  (FAA,  1996)  Chapter  5  represents  one  part  of  a  larger  scale  effort  to  keep 
the  guidance  in  the  entire  HFDG  current.  The  revised  Chapter  5  will  be  incorporated  in  the 
future  release  of  a  new  HFDG  CD-ROM. 

The  revised  version  of  Chapter  5  is  presented  in  Appendix  A.  A  table  of  contents  precedes  the 
document.  The  guidelines  are  followed  by  a  glossary  containing  key  terms,  a  list  of  references, 
and  an  index  of  keywords  that  can  be  used  to  find  information  in  the  document. 
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5.0  Automation 

Definitions.  Automation  is  the  independent  accomplishment  of 
a  function  by  a  device  or  system  that  was  formerly  carried  out 
by  a  human.  [Source:  NRC  1998;  Parasuraman  &  Riley,  1997] 


5.1  General 

D  5.1.1  Minimum  automation  human  factors  requirements.  An 

automated  system  should 

a.  provide  sufficient  information  to  keep  the  user  informed 
of  its  operating  mode,  intent,  function,  and  output; 

b.  inform  the  user  of  automation  failure  or  degradation; 

c.  inform  the  user  if  potentially  unsafe  modes  are  manually 
selected; 

d.  not  interfere  with  manual  task  performance;  and 

e.  allow  for  manual  override.  [Source:  Veridian  (AHCI),  1998; 
Billings,  1997] 

■  5.1.2  User  in  command.  Automated  systems  shall  prevent  the 

removal  of  the  user  from  the  command  role.  [Source:  Billings, 
1997] 


Discussion.  The  reasoning  behind  this  rule  is  twofold. 
First,  it  is  ultimately  the  user  who  is  responsible  for  the 
task.  Second,  automation  is  subject  to  failure.  Therefore,  it 
is  the  user,  not  the  automation  who  must  be  in  control  of 
the  system  with  the  automation  playing  a  subservient  role. 
[Source:  Billings,  1997] 

■  5.1.3  Automate  only  to  improve  performance.  Functions  shall 

be  automated  only  if  they  improve  system  performance  without 
reducing  human  involvement,  situation  awareness,  or  human 
performance  in  carrying  out  the  intended  task.  [Source:  Billings, 
1991] 


Discussion.  The  introduction  of  automation  is  often 
intended  to  reduce  workload  and  augment  performance; 
however,  this  is  not  always  the  result.  Automation  can 
lead  to:  distraction  from  the  primary  task,  increased 
workload,  boredom,  or  complacency.  Automation  can 
also  have  psychosocial  impacts,  influencing  job 
satisfaction  or  self  worth.  [Source:  Bowers,  Deaton,  Oser, 
Prince  &  Kolb,  1995;  Danaher,  1980;  Edwards,  1976; 
Parasuraman,  Molloy,  Mouloua,  &  Hilburn,  1996;  Wiener,  1989; 
Wiener  &  Curry,  1980] 
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D  5.1.4  Human-centered  automation.  Automation  should  be  used 
to  support  the  user(s)  where  appropriate  (human-centered 
automation),  not  implemented  simply  because  the  technology  is 
available  (technology-centered  automation).  [Source:  Billings, 

■  5.1.5  Enabling  users  to  carry  out  tasks.  Automation  shall  help 
or  enable  the  users  to  carry  out  their  responsibilities  and  tasks 
safely,  efficiently,  and  effectively.  [Source:  Billings,  1991] 

Discussion.  Carrying  out  a  task  effectively  means 
producing  the  desired  result.  Carrying  out  a  task 
efficiently  means  that  the  desired  result  is  produced  with 
a  minimum  of  waste  (usually  in  relation  to  time). 

■  5.1.6  Clear  relationship  with  user  tasks.  The  relationships 
between  display,  control,  decision  aid,  and  information  structure 
and  user  tasks  and  functions  shall  be  clear  to  the  user.  [Source: 
Nuclear  Regulatory  Commission  (NUREG-700),  1996;  Nuclear  Regulatory 
Commission  (NUREG/CR-6105),  1994] 

Discussion.  The  user  needs  to  be  able  to  see  clearly  how 
the  display  or  decision  aid,  and  so  on,  facilitates  the 
completion  of  the  necessary  task. 

■  5.1.7  Active  involvement  in  operation.  Users  shall  be  given  an 
active  role  through  relevant  and  meaningful  tasks  in  the  operation 
of  a  system  regardless  of  the  level  of  automation  being  employed. 
[Source:  AHCI,  1998;  Billings,  1991] 


Discussion.  User  awareness  of  system  state  cannot  be 
sustained  passively.  Active  involvement  is  essential  for 
operators  to  exercise  their  responsibilities  and  be  able  to 
respond  to  emergencies.  Reducing  active  involvement 
may  be  detrimental  to  the  user’s  understanding  of 
important  information,  may  lead  to  longer  response  times 
in  case  of  emergencies,  or,  in  the  long  term,  may  lead  to 
loss  of  relevant  knowledge  or  skills.  [Source:  Galster, 

Duley,  Masalonis,  &  Parasuraman,  2001;  Garland  &  Hopkin,  1994- 
Hopkin,  1988;  Sarter  &  Woods,  1992  (as  found  in  Scerbo,  1996); 
Wickens,  1992  (as  found  in  Scerbo,  1996)] 

D  5.1.8  Appropriate  to  user  expertise.  Procedures  employed  in 
automation  should  be  appropriate  to  the  user’ s  level  of  expertise 
with]  the  system.  [Source:  Defense  Information  Systems  Agency  (DISA), 

Example.  Shortcuts  such  as  function  keys  can  be 
provided  for  the  more  experienced  users,  whereas  novice 
users  can  still  use  standard  procedures. 
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D  5.1.9  Implementing  automation.  How  automation  is 

implemented  should  be  determined  by  the  explicit  goals  of  the 
system,  not  by  comparison  between  automated  and  manual 
systems.  [Source:  Wiener  &  Curry,  1980] 

Discussion.  When  automation  is  implemented,  explicit 
goals  of  the  system  need  to  be  kept  in  mind,  thus,  an 
automated  system  does  not  need  to  perform  a  task  the 
same  way  as  it  was  performed  manually  to  be  effective. 

°  5.1.10  Demands  for  cognitive  resources.  Automation  should 

not  increase  the  demands  for  cognitive  resources  (thinking  or 
conscious  mental  processes).  [Source:  Bainbridge,  1983;  Parasuraman 
&  Riley,  1997;  Wiener  &  Curry,  1980;  Woods,  1996] 

Discussion.  Automation  that  increases  the  demand  for 
cognitive  resources  is  considered  clumsy.  Expert  users  in 
complex,  dynamic  systems  have  been  observed  to  cope 
with  clumsy  automation  by  using  only  a  subset  of  the 
available  functionality,  especially  during  periods  of  high 
workload.  [Source:  Woods,  1996] 

D  5.1.11  Avoid  extreme  workload  levels.  Extreme  levels  of 
workload  (low  or  high)  due  to  automation  use  should  be 
avoided.  [Source:  Hilburn,  Jorna,  Byrne,  &  Parasuraman,  1996;  NRC, 
1993;  Warm,  Dember,  &  Hancock,  1996;  Wiener,  1988] 

Discussion.  Extreme  levels  of  workload  can  be  caused 
by  clumsy  automation.  Clumsy  automation  can  cause 
extreme  workload  levels  by  increasing  workloads  when 
they  are  already  high  (e.g.,  for  pilots,  during  the  high 
workload  flight  phases  of  take-off  and  landing)  and 
decreasing  workloads  that  are  already  low  (e.g., 
providing  a  pilot  with  the  ability  to  engage  autopilot 
during  the  low  workload  “cruise”  phase  of  a  flight). 
Automation  is  often  introduced  to  reduce  workload. 
However,  reduction  of  workload  may  not  always  be 
advantageous,  for  example,  if  workload  is  already  low. 
[Source:  Hilburn  et  al.,  1996;  Parasuraman  &  Mouloua,  1996] 

■  5.1.12  Distraction  from  operations.  User  interaction  with 

automation  shall  not  require  the  user  to  take  significant  amounts 
of  attention  away  from  the  primary  task.  [Source:  Danaher,  1980] 

Discussion.  When  automation  requires  the  user  or  one 
member  of  the  user  team  to  devote  a  significant  amount 
of  attention  to  adjusting  or  monitoring  the  automation, 
this  removes  the  user  away  from  minute-to-minute 
operations,  thereby  taking  the  user  out  of  the  loop.  This 
can  be  especially  dangerous  if  an  abnormal  situation 
occurs  that  needs  to  be  remedied  quickly.  [Source: 

Danaher,  1980] 
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5.1.13  Inappropriate  timing.  Automation  should  not  interrupt 
at  inappropriate  times  such  as  during  periods  of  high  workload 
or  during  critical  moments  in  a  process.  [Source:  Woods,  1996] 

Discussion.  An  interruption  during  high  workload  or  at 
a  critical  moment  can  cause  a  delay  in  the  user’s  ability 
to  respond  to  a  malfunction,  leading  to  a  potential 
failure.  If  the  user  is  attending  to  a  malfunction  in  an 
automated  task  and  is  interrupted,  the  interruption 
depletes  the  user’ s  resources  causing  him  to  be  less 
capable  of  averting  the  potential  failure.  For  example,  in 
the  cockpit,  certain  automation  functions  might  be 
stopped  from  interrupting  during  the  takeoff  and  landing 
portions  of  flight. 

5.1.14  Easier  to  perform.  An  automated  task  should  be  less 
difficult  to  perform  than  the  manual  task  it  replaces.  [Source: 
AHCI,  1998] 

5.1.15  Guided  use  of  automation.  Standard  operating 
procedures  and  company  policies  should  guide  users  in  the 
appropriate  use  of  automation,  although  the  user  should  be 
ultimately  responsible  to  make  the  decision  to  use  or  not  use  the 
automation.  [Source:  Billings,  1997;  Parasuraman  &  Riley,  1997] 

5.1.16  Easy  data  access.  Data  that  are  needed  by  the  user  shall 
be  easily  accessible.  [Source:  NUREG/CR-6105,  1994;  NUREG-0700, 
1996] 

Discussion.  User  requirements  can  serve  as  a  guide  of 
whether  the  data  are  available  at  all  times,  accessible  at 
the  users’  discretion,  or  not  at  all  if  the  user  does  not 
need  information. 

5.1.17  Data  entry  format.  The  automated  system  should 
prompt  users  as  to  the  correct  data  entry  format.  [Source:  Billings, 

Example.  If  the  automated  system  requires  that  the  data 
be  entered  in  all  capital  letters,  it  should  specifically  tell 
the  user  to  enter  the  data  in  capital  letters. 

5.1.18  Error  resistance  and  error  tolerance.  Automation 
should  be  error  resistant  and  error  tolerant.  [Source:  Billings,  1991] 

Discussion.  To  make  a  system  error  resistant  is  to 
make  it  difficult  for  a  user  to  make  an  error.  Simplicity 
in  design  and  the  provision  of  clear  information  are  tools 
to  improve  error  resistance.  Error  tolerance  is  the 
ability  to  mitigate  the  effects  of  human  errors  that  are 
committed.  Error  tolerance  can  be  improved  by  adding 
monitoring  capabilities  to  the  automation.  Electronic 
checklists  also  have  the  potential  to  improve  error 
resistance  by  providing  reminders  of  items  that  need  to 
be  completed.  [Source:  Billings,  1991] 
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■  5.1.19  Predictable  automated  systems.  Automated  systems 

shall  behave  predictably  so  that  the  user  knows  the  purpose  of  the 
automation  and  how  the  operation  will  be  affected  by  that 
automation.  [Source:  Billings,  1991,  1996] 

Discussion.  The  predictability  of  an  automated  system 
allows  the  user  to  know  what  to  expect  when  the 
automation  is  functioning  correctly.  This  makes  it  easier 
for  the  user  to  recognize  when  the  system  is  not 
functioning.  [Source:  Billings,  1996] 


■  5.1.20  Ensure  safe  operations  are  within  human  capacity. 

Systems  shall  not  be  so  reliant  on  automation  or  on  human  skills 
degraded  by  automation  use  that  human  users  can  no  longer 
safely  recover  from  emergencies  or  operate  the  system  manually 
if  the  automation  fails.  [Source:  Billings,  1996;  NRC,  1998] 


Discussion.  A  balance  is  needed  between  the  efficiency 
created  by  automation,  the  need  for  the  operator  to  be 
able  to  recover  from  emergencies,  and  control  the  system 
manually  in  case  the  automation  fails. 

D  5.1.21  Veto  of  user  actions.  The  automation  should  not  be  able 
to  veto  user  actions  leaving  the  user  without  means  to  override 
or  violate  the  rules  that  govern  the  automation  unless  there  is 
not  enough  time  for  the  user  to  make  a  decision.  [Source:  Garland 
&  Hopkin,  1994;  Inagaki,  1999] 

Discussion.  Problems  with  automation  can  occur  when 
the  automated  options  do  not  apply  to  a  situation  and  the 
user  is  restricted  to  the  options  provided  by  the 
automation. 


■  5.1.22  Interaction  consistency.  The  way  that  automation 

systems  interact  with  their  users  shall  reflect  a  high  degree  of 
consistency  within  and  between  systems.  [Source:  NUREG-700, 
1996] 


Discussion.  There  are  many  possible  types  of 
interaction,  such  as  menu  selection,  direct  manipulation, 
and  form-filling.  (See  Revised  Chapter  8  on  computer- 
human  interfaces  for  more  information  on  interaction). 
An  example  of  inconsistent  interaction  would  be  having 
one  system  require  filling  in  forms  as  the  interaction 
method,  whereas  another  system  requires  menu-driven 
interaction. 
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D  5.1.23  Easy  to  understand  and  use.  Automated  systems  and 
associated  integrated  information  displays  should  be  intuitive, 

easy  to  understand,  and  easy  to  use.  [Source:  Billings,  1991;  Sarter 
&  Woods,  1994;  Woods,  1996] 

Discussion.  System  operations  that  are  easily 
interpretable  or  understandable  by  the  user  can  facilitate 
the  detection  of  improper  operation  and  the  diagnosis  of 
malfunctions.  [Source:  Wiener  &  Curry,  1980] 

D  5.1.24  Simple  to  learn.  Automation  should  be  simple  for  the 
users  to  learn.  [Source:  Billings,  1991;  Wiener  &  Curry,  1980] 

D  5.1.25  Input  and  setup.  Automated  systems  should  provide  a 
way  to  check  automation  setup  and  to  check  information  used  as 
input  for  the  automated  system.  [Source:  Wiener  &  Curry,  1980; 
Wickens,  2000] 

Discussion.  Automation  failures  are  often  due  to  setup 
error.  Although  the  automated  system  itself  could  check 
some  of  the  setup,  independent  error-checking  equipment 
or  procedures  may  be  needed.  The  user  needs  to  be  able 
to  distinguish  whether  a  failure  occurred  due  to  the 
automation  setup  or  due  to  an  inaccuracy  in  the  input 
information.  An  automation  failure  could  have  been 
caused  by  a  malfunction  of  an  algorithm  or  by  the  input 
of  inaccurate  data.  For  example,  if  the  automated  system 
relies  on  primary  radar  and  secondary  radar  as  inputs  and 
uses  an  algorithm  to  predict  conflicts,  a  failure  could 
arise  from  faulty  data  from  either  the  primary  or 
secondary  radar  or  from  the  algorithm  that  combines  this 
information.  [Source:  Wiener  &  Curry,  1980;  Wickens,  2000] 


5.2  Design  and  evaluation 


D  5.2.1  User  involvement  in  design.  Users  should  be  involved  in 
the  design  of  an  automated  tool.  [Source:  Amalberti,  1999;  Billings, 
1997;  Parasuraman,  Sheridan,  &  Wickens,  2000] 

Discussion.  Input  from  the  user  is  essential  in  defining 
information  requirements. 


D  5.2.2  Design  based  on  human-centered  goals  and  functions. 
Design  of  automation  should  begin  by  choosing  the  human- 
centered  criteria  (goals)  of  the  system  and  then  defining  the 
functions  that  the  system  will  perform.  [Source:  Wiener  &  Curry 
1980] 


Discussion.  Defining  the  goals  and  functions  of  an 
automated  system  may  require  the  use  of  task  analysis. 
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■  5.2.3  Effect  on  coordination.  When  new  automation  is 
introduced,  the  designers  shall  consider  the  possibility  of 
negative  effects  on  team  coordination.  [Source:  Wiener,  1989] 

Discussion.  Automation  may  deplete  team  interaction 
and  cooperation  unless  all  parties  are  provided  with 
information  that  allows  them  to  be  actively  involved  in 
the  task.  Automation  can  cause  physical  difficulty  in 
seeing  what  the  other  team  member  is  doing,  reduce  the 
ability  to  cross  monitor,  change  traditional  roles  and 
responsibilities,  and  change  the  manner  in  which  team 
members  attempt  to  help  one  another.  [Source:  Danaher, 
1980;  Rudisill,  1994] 

■  5.2.4  Assess  overall  impact.  The  overall  impact  of  automation 
shall  be  thoroughly  examined  before  implementation  to  ensure 
that  changes  do  not  result  in  additional  complexities,  loss  of 
situational  awareness,  or  possibilities  for  error.  [Source:  Woods, 
1996] 


Discussion.  Automation  of  some  user  tasks  may  result 
in  the  user  processing  less  information  or  processing 
information  at  less  depth.  A  diminished  understanding 
and  appreciation  for  the  overall  situation  may  result. 
[Source:  Garland  &  Hopkin,  1994] 

D  5.2.5  Validation.  Contextually  valid  human-in-the-loop 

experiments  and  simulations  should  be  conducted  to  validate  and 
refine  automated  system  design.  [Source:  NRC,  1998] 

■  5.2.6  Interaction  with  other  functions.  Possible  interactions 
with  other  tools,  system  functions,  and  user  tasks  shall  be 
evaluated  when  new  automation  is  designed.  [Source:  NRC,  1998] 

■  5.2.7  Integration.  New  automation  components  shall  be  tested 
with  the  complete  system,  including  other  automated 
components  of  the  system,  to  ensure  they  function  together  as  an 
effective  whole.  [Source:  NRC,  1998] 

■  5.2.8  Testing  in  normal  and  failure  modes.  Automated 
systems  shall  be  tested  under  normal  modes  of  operation  and 
under  failure  modes  of  the  automation.  [Source:  NRC,  1998; 
Wickens,  2000] 

■  5.2.9  Test  before  implementation.  Automated  systems  shall  be 
tested  in  a  realistic  operational  environment  with  representative 
users  before  implementation  to  ensure  that  operator  performance  is 
not  compromised  and  workload  is  not  increased.  [Source:  Drury, 
1998] 
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5.3  System  response  and  feedback 

D  5.3.1  Visualize  consequences  of  decisions.  The  user  should  be 
able  to  visualize  the  consequences  of  a  decision,  whether  made 
by  the  user  or  the  automated  system.  [Source:  Billings,  1996] 

0  5.3.2  Command  response.  Automated  system  responses  to  user 

commands  should  be  brief  and  unambiguous.  [Source:  Billings, 

D  5.3.3  User  awareness  of  function.  The  automated  system 

should  keep  the  user  aware  on  a  continuing  basis  of  the  function 
(or  malfunction)  of  each  automated  system  and  the  results  of 
that  function  (or  malfunction).  [Source:  Billings,  1996] 

D  5.3.4  Feedback.  Automation  should  provide  the  user  with 
effective  feedback  on  its  actions  and  the  purpose  of  these 
actions.  [Source:  Woods,  1996] 

Discussion.  When  feedback  is  poor,  automation  is 
considered  silent.  Silent  automation  may  result  in 
coordination  and  system  failures.  Users  may  be 

surprised  by  the  behavior  of  silent  automation.  [Source: 
Woods,  1996] 


5.4  Interface 

D  5.4.1  Interface  simplicity.  The  automation  interfaces  should 
represent  the  simplest  design  consistent  with  functions  and  tasks  of 
the  users.  [Source:  NUREG-700,  1996] 

Discussion.  Simplicity  for  the  user  is  achieved  by 
attaining  compatibility  between  the  design  and  human 
perceptual,  physical,  cognitive,  and  dynamic  motor 
responsiveness  capabilities.  (See  HFDG  Update: 

Revision  to  Chapter  8  (Ahlstrom  &  Longo,  2001)  on 
computer-human  interfaces  for  more  information  on 
interface  design.)  [Source:  NUREG-700,  1996] 

■  5.4.2  Interface  consistency.  Human  interfaces  in  automation 

programs  and  systems  shall  have  a  high  degree  of  consistency. 
[Source:  NUREG-700,  1996]  J 

Discussion.  Consistency  can  be  obtained  by  presenting 
information  in  predictable  locations  and  keeping  elements 
of  screens  such  as  headers,  fields,  and  labels  consistent  in 
appearance  and  relative  location  throughout  a  system  or 
application.  (See  HFDG  Update:  Revision  to  Chapter  8 
(Ahlstrom  &  Longo,  2001)  on  computer-human  interfaces  for 
more  information  on  interface  design.)  [Source: 

Shneiderman,  1998] 
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5.4.3  Consistent  with  user  expectations.  Automated  systems 
and  interfaces  should  be  consistent  with  the  expectations  and 
understandings  of  users.  [Source:  Billings,  1991,  1996] 

5.4.4  Logical  interface  structure.  Automation  interfaces  shall 
reflect  an  obvious  logic  based  on  user  task  needs  and 
capabilities.  [Source:  NUREG-6105,  1994;  NUREG-700,  1996] 

5.4.5  Location  status.  Interfaces  and  navigation  aids  shall  make 
it  easy  for  users  to  know  where  they  are  in  the  data  space.  [Source: 
NUREG-6105,  1994;  NUREG-700,  1996] 

5.4.6  Use  spatial  representations  where  possible.  Where 
possible,  spatial  representations  of  information  should  be  used 
instead  of  verbal  or  textual  displays  in  high  workload  situations. 
[Source:  Barnes,  1981] 

Discussion.  Although  humans  are  often  better  able  to 
attend  to  spatial  representations,  it  is  not  always  easy  or 
even  possible  to  create  spatial  representations  of 
information.  [Source:  Barnes,  1981] 

5.4.7  Dynamic  information.  Dynamic  information  (information 
that  changes  over  time)  should  be  presented  in  real  time  and  on 
demand  to  ensure  accurate  and  timely  decision-making.  [Source: 
Morris,  Rouse,  &  Ward,  1985] 
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5.5  User  acceptance  and  trust 

D  5.5.1  User  trust  in  automation.  To  increase  user  trust  in 
automation,  automation  performance  should  be 

a.  reliable  and  predictable  with  minimal  errors, 

b.  robust  (able  to  perform  under  a  variety  of 
circumstances), 

c.  familiar  (use  terms  and  procedures  familiar  to  the  user), 
and 

d.  useful.  [Source:  Lee  &  Moray,  1992;  Lerch  &  Prietula,  1989; 
Masalonis  &  Parasuraman,  1999;  Muir,  1987  (as  found  in  Riley 
1996);  NRC,  1998] 


Discussion.  Trust  in  automation  tends  to  be  relatively 
stable.  However,  changes  in  trust  may  occur  over  time. 
User  trust  in  automation  can  increase  with  reliable  and 
predictable  performance.  Decreases  in  trust  may  occur 
as  a  result  of  some  critical  error  or  automation  failure.  It 
is  more  difficult  for  users  to  regain  trust  in  automation 
after  a  failure  than  to  develop  an  initial  trust.  Higher 
trust  in  automation  is  not  always  better  because 
automation  errors  may  be  overlooked  due  to 
complacency.  Decreases  in  trust  typically  occur 
suddenly,  but  increases  happen  slowly  and  steadily.  The 
consequences  of  an  automation  failure  (e.g  the 
magnitude  of  an  error)  impact  the  decline  in  trust. 

[Source:  Lee  &  Moray,  1992;  Lerch  &  Prietula,  1989;  Masalonis  & 
Parasuraman,  1999;  Riley,  1996;  NRC  1998] 

D  5.5.2  Trust  calibration.  Training  should  be  provided  to  enable 
the  user  to  calibrate  their  trust  in  the  automated  system.  [Source: 
Cohen,  Parasuraman,  &  Freeman,  1998] 

Discussion.  Training  will  allow  the  user  to  develop  an 
adequate  model  of  how  reliable  or  unreliable  the 
automation  is  under  specific  conditions. 

D  5.5.3  Availability  of  automation.  The  automated  system  should 
be  available  to  the  user  as  needed.  [Source:  Morris,  Rouse,  &  Ward 
1985] 

■  5.5.4  Noninterference  with  user  tasks.  The  automated  system 
shall  not  interfere  with  task  performance.  [Source:  Andes,  1987] 

Discussion.  A  user  will  be  less  likely  to  accept  an 
automated  system  that  interferes  with  their  ability  to 
perform  tasks.  [Source:  Andes,  1987] 
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■  5.5.5  Accurate  and  reliable  information.  Automation  shall 

provide  accurate  and  reliable  information.  [Source:  Andes,  1987] 

Discussion.  When  users  believe  automation  to  be  highly 
reliable,  they  place  greater  trust  in  it.  However,  there  is 
a  trade-off  involved  with  a  constant  high  level  of 
automation  reliability  and  predictability.  Constant  high 
levels  of  automation  reliability  and  predictability  may  be 
more  likely  to  promote  complacency  and  may  cause  users 
to  monitor  automation  with  less  vigilance.  [Source: 
Dzindolet,  Pierce,  Beck,  &  Dawe,  1999;  Parasuraman,  Molloy,  & 
Singh,  1993,  (as  found  in  Masalonis  &  Parasuraman,  1999); 

Wiener,  1981] 

D  5.5.6  Minimize  changes  due  to  automation.  Changes  in 
cognitive  processing,  ways  of  thinking,  and  methods  and  skills 
used  for  new  automation  should  be  minimized.  [Source:  Garland  & 
Hopkin,  1994] 

Discussion.  Automation  that  requires  different  kinds  of 
cognitive  processing,  ways  of  thinking,  and  discarding  of 
traditional  methods  and  skills  may  cause  the  system  to  be 
both  less  efficient  and  less  acceptable  to  the  users.  This 
could  include  automatic  conversion  of  data  into  a  usable 
format.  [Source:  Garland  &  Hopkin,  1994] 

D  5.5.7  Understanding  of  automation  function.  To  promote  user 
acceptance  of  automation,  users  should  be  taught  how  an 
automated  system  functions.  [Source:  Cohen  et  al.,  1998;  Dzindolet  et 
al.,  1999;  Lehner,  Mullin,  &  Cohen,  1989] 

Discussion.  The  better  the  user  understands  the 
automation,  the  more  likely  the  user  is  to  trust  the 
automation  appropriately.  Designers  need  to  alter  the 
false  belief  that  automation  is  perfect  and  ensure  that  the 
users  understand  when  the  automation  is  likely  to  become 
unreliable.  [Source:  Dzindolet  et  al.,  1999] 

5.6  Modes 

°  5.6.1  Clear  mode  and  function  identification.  When  control, 

display,  or  automation  functions  change  in  different  modes  of 
operation,  mode  and  function  identification  and  status  should  be 
clear.  [Source:  Billings,  1991;  Sarter  &  Woods,  1995] 

Discussion.  Lack  of  effective  feedback  on  the  state  of 
automation  (including  which  mode  is  active)  can  lead  to 
mode  errors.  [Source:  Sarter  &  Woods,  1995] 
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D  5.6.2  Identification  of  alternatives  in  rarely  used  modes. 

Seldom-used  modes  and  functions  should  be  clearly  identified. 
[Source:  Billings,  1991] 

Example.  As  automated  systems  become  more  complex 
with  many  modes  and  functions,  the  cognitive  burden 
caused  by  the  need  for  mode  awareness  increases. 
Seldom-used  modes  and  functions  will  pose  the  largest 
burden  on  the  user  because  of  a  lack  of  familiarity. 
Enabling  the  user  to  immediately  recognize  the  purpose 
of  modes  and  functions,  such  as  labeling  the  engine 
failure  function  “ENG  OUT”,  can  lessen  this  burden. 
[Source:  Billings,  1997] 

°  5.6.3  Mode  accessibility.  Frequently  used  modes  should  be 

more  accessible  than  infrequently  used  modes.  [Source:  AHCI 
1998] 

D  5.6.4  Number  of  modes.  The  number  of  different  modes  for  a 
given  system  should  be  minimized.  [Source:  AHCI,  1998] 

Discussion.  Multiple  modes  will  provide  a  means  of 
flexibility  but  will  introduce  more  opportunities  for 
error.  Furthermore,  automation  that  has  multiple  modes 
of  operation  can  be  difficult  to  learn  and  can  produce 
increases  in  workload.  Users  must  understand  and 
remember  how  and  when  to  use  each  mode,  and  they 
must  remember  which  mode  is  currently  active.  [Source: 
Scerbo,  1996;  Woods,  1996] 

°  5.6.5  Switching  between  modes.  The  user  should  be  able  to 

easily  switch  between  modes.  [Source:  AHCI,  1998] 

D  5.6.6  Consistent  features  and  functions.  Features  and 
functions  that  are  common  between  display  modes  should  be 
consistent.  [Source:  AHCI,  1998] 

Discussion.  In  the  original  Standard  Terminal  Automation 
Replacement  System  (STARS),  the  Full  Service  Level 
(FSL)  and  the  Emergency  Service  Level  (ESL)  had 
independent  and  inconsistent  interfaces  requiring  users  to 
learn  two  different  interfaces:  mouse  interaction  styles  and 
status-coding  schemes.  This  can  lead  to  additional  training 
requirements  and  workload.  The  human  factors  team 
recommended  that  the  two  subsystems  have  identical 
coding  strategies,  identical  access  and  execution  of  system 
commands,  consistent  data  display  formatting,  and 
consistent  monitoring  and  reporting  of  resources.  [Source: 
Standard  Terminal  Automation  Replacement  System  Human  Factors 
Team,  1997,  1998] 

D  5.6.7  Interactions  between  modes.  The  automated  system 
should  alert  the  user  to  the  implications  of  interactions  between 

modes,  especially  when  they  are  potentially  hazardous.  [Source: 
Billings,  1996]  1 
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5.7  Monitoring 


5.6.8  Alert  to  unsafe  modes.  The  automated  system  should 
either  prevent  the  use  of  potentially  unsafe  modes  or  alert  the 
user  that  a  particular  mode  may  be  hazardous.  [Source:  Billings, 
1996] 


5.7.1  Monitoring  automated  systems.  The  system  shall  be 
designed  so  that  users  are  able  to  monitor  the  automated  systems 
and  the  functionality  of  its  hardware  and  software,  including  the 
display  of  status  and  trend  information,  when  necessary.  [Source: 
Billings,  1991] 

Discussion.  One  way  that  this  can  be  accomplished  is  by 
providing  the  user  with  access  to  raw  data  that  the 
automation  processes. 

5.7.2  Monitoring  changing  data.  Changing  data  that  must  be 
monitored  by  the  users  should  be  displayed  in  a  graphic  format. 
[Source:  Smith  &  Mosier,  1986] 

5.7.3  Active  control  and  monitoring.  Automation  should  be 
designed  so  that  users  are  involved  in  active  control  and 
monitoring  rather  than  just  passive  monitors.  [Source:  Hilburn, 
Jorna,  &  Parasuraman,  1995;  Wickens  &  Kessel,  1979] 

Discussion.  Automation  failures  may  be  easier  to  detect 
when  users  are  involved  in  both  active  control  and 
monitoring,  than  when  they  are  just  passive  monitors. 
[Source:  Hilburn  et  al.,  1995;  Wickens  &  Kessel,  1979] 

5.7.4  Monitoring  resources.  System  designers  should  allow 
adequate  cognitive  resources  for  monitoring  by  ensuring  that 
task  load  does  not  become  excessive.  [Source:  Wiener  &  Curry, 
1980] 


Discussion.  Users  of  automated  systems  may  experience 
higher  levels  of  mental  workload  than  manual  controllers 
due  to  monitoring,  diagnosis,  and  planning,  with 
significant  cognitive  demand  resulting  from  relatively 
“simple”  vigilance  tasks.  [Source:  Deaton  &  Parasuraman, 
1993;  Sheridan,  1970;  Warm  et  al.,  1996] 

5.7.5  Limited  monitoring  time.  Users  should  not  be  required  to 
perform  purely  monitoring  tasks  for  longer  than  20  minutes  at  a 
time.  [Source:  Parasuraman  et  al.,  1993;  Warm  et  al.,  1996] 

Discussion.  Users  may  become  complacent  in 
monitoring  automated  systems  if  they  have  other  tasks  to 
complete  simultaneously.  Such  decrements  in  user 
monitoring  of  automated  systems  have  been  observed  to 
occur  in  the  laboratory  in  as  little  as  20  minutes.  [Source: 
Parasuraman  et  al.,  1993;  Warm  et  al.,  1996] 
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D  5.7.6  Display  integration.  When  users  must  monitor  multiple 
displays,  important  events  should  occur  in  the  same  display  in 
order  to  promote  effective  monitoring  performance.  TSource: 
Warm  et  al.,  1996] 

0  5.7.7  Spatial  certainty.  Important  events  should  occur  in  the 

same  location  on  a  display  in  order  to  promote  effective 
monitoring  performance.  [Source:  Warm  et  al.,  1996] 

Discussion.  Users  will  be  able  to  detect  a  particular 
event  more  easily  if  they  know  where  that  event  will 
occur  (i.e.,  spatial  certainty).  Spatial  uncertainty  has 
been  shown  to  increase  perceived  workload  and  decrease 
performance  efficiency.  If  users  do  not  know  where  on  a 
display  an  event  will  occur  then  they  must  engage  in 
visual  scanning  to  look  for  the  event.  [Source:  Adams  & 
Boulter,  1964;  Milosevic,  1974  (as  found  in  Warm  et  al.,  1996)] 

D  5.7.8  Indication  of  monitoring.  Automated  systems  that  are 
without  incident  for  long  periods  of  time  should  provide  some 
type  of  indication  that  the  automation  is  still  monitoring  the 
system.  [Source:  AHCI,  1998] 

D  5.7.9  Monitoring  human  interactions.  Automated  systems 
should  be  able  to  monitor  user  interactions  and  to  warn  of  user 
errors.  [Source:  Billings,  1991] 

Discussion.  To  monitor  user  interactions  and  to  warn  of 
user  errors,  automated  systems  may  need  to  be  able  to 
receive  input  information  on  user  intentions. 

0  5.7.10  Monitoring  of  critical  functions.  Critical  automation 

functions  should  be  independently  monitored  by  the  user. 

[Source:  Billings,  1996] 

Definition.  A  critical  function  is  a  function  that  can 
cause  system  failure  when  a  malfunction  is  not  attended 
to  immediately. 

Discussion.  When  a  function  is  critical,  combining  the 
monitoring  of  that  critical  function  with  other,  possibly 
less  critical  functions  may  lead  to  delays  in  response. 
When  a  critical  function  is  independently  monitored,  a 
user  can  respond  to  a  malfunction  very  quickly  (within 
one  second).  If  a  user  is  attending  to  another  task  when 
there  is  a  malfunction,  there  will  be  a  delay  in  the  user’s 
response  (several  seconds).  In  this  period  of  delayed 
response,  the  malfunction  can  cause  the  system  to  fail. 

For  this  reason,  critical  functions  require  constant 
attention.  Critical  automation  functions  do  assist  in  the 
completion  of  critical  tasks,  however  they  do  not  assist  in 
freeing  the  user  to  attend  to  other  tasks.  [Source: 
Parasuraman  et  al.,  1996] 
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a  5.7.11  Adequate  mental  model.  Users  should  be  given  an 
adequate  understanding  of  how  the  automated  system  works  in 
order  to  monitor  effectively.  [Source:  Carroll  &  Olsen,  1988  (as  found 
in  Scerbo,  1996);  Wickens,  1992  (as  found  in  Scerbo,  1996);  Wickens  & 
Flach,  1988;  Woods,  1994  (as  found  in  Scerbo,  1996);  Woods,  1996] 

Discussion.  Users  must  possess  accurate  mental  models 
of  automated  systems  in  order  to  monitor  effectively, 
comprehend  current  situations,  plan  their  actions,  predict 
future  system  states,  remember  past  instructions,  and 
diagnose  system  failures.  One  way  to  establish  adequate 
mental  models  is  through  training.  [Source:  Scerbo,  1996; 
Wickens,  1992  (as  found  in  Scerbo,  1996);  Wickens  &  Flach,  1988; 
Woods,  1994  (as  found  in  Scerbo,  1996);  Woods,  1996] 

0  5.7.12  Intermittent  manual  control.  Intermittent  periods  of 

manual  control  should  be  used  during  extended  periods  of  task 
automation  to  improve  monitoring  of  the  automation.  (See 
adaptive  automation  section  5.13.)  [Source:  Morrison,  Cohen,  & 
Gluckman,  1993;  Parasuraman  et  al.,  1993] 

Discussion.  Complacency  is  a  major  concern  with 
automation.  Intermittent  periods  of  manual  control  have 
been  advocated  as  a  means  of  minimizing  complacency. 
Automation  may  also  result  in  the  decrement  of  cognitive 
abilities  such  as  instrument  scan  and 
navigation/positional  [situation]  awareness  and  the  loss  of 
manual  skills,  making  transitions  from  automated  to 
conventional  systems  difficult.  Because  automation  can 
decrease  basic  manual  skills,  these  skills  should  be  used 
and  maintained.  Intermittent  periods  of  manual  control 
during  which  automation  is  suspended  periodically  can 
promote  optimal  user  performance,  and  allow  better 
recovery  from  failure,  regardless  of  the  type  of  task  that 
is  automated.  [Source:  Endsley  &  Kiris,  1995;  Morrison  et  al., 
1993;  Rudisill,  1994;  Wickens,  1992  (as  found  in  Scerbo,  1996)] 

Q  5.7.13  Minimize  noise.  Environmental  noise  should  be 

minimized  to  ensure  optimal  vigilance.  [Source:  Warm  et  al.,  1996] 

Discussion.  Vigilance  will  be  reduced  when  high  levels 
of  intermittent  noise  are  present  in  the  environment, 
especially  if  the  information  processing  task  demands  are 
high.  Noise  is  defined  as  sounds  that  are  loud, 
disagreeable  or  unwanted.  Music,  however,  may  act  as  a 
stimulant  and  offset  decrements  in  arousal  due  to  fatigue 
and  prolonged  performance.  [Source:  Davies,  Lang  & 
Shackleton,  1973;  Hancock,  1984  (as  found  in  Warm  et  al.,  1996); 
Matthews,  Davies,  Westerman,  &  Stammers,  2000] 
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D  5.7.14  Circadian  rhythm  effects  on  performance.  System 
designers  should  consider  the  effects  of  circadian  rhythms  on 
user  vigilance  and  monitoring  performance.  [Source:  Colquhoun 
1977  (as  found  in  Warm  et  al.,  1996)] 

Discussion.  It  will  be  most  difficult  for  users  to  maintain 
monitoring  performance  during  the  early  morning  (8:00 
a.m.)  when  body  temperature  is  low.  Performance  will 
peak  late  in  the  day  (between  5:00  p.m.  and  9:00  p.m.)  as 
body  temperature  rises.  Monitoring  performance  will  then 
decline  again  as  body  temperature  drops.  Maintaining 
monitoring  performance  can  also  be  difficult  for  users 
working  irregular  work  schedules.  Working  consecutive 
night  shifts,  prolonged  work  shifts,  or  starting  hours  that 
are  too  early  can  cause  users  to  experience  a 
desynchronization  of  their  circadian  rhythm  caused  by  the 
accumulation  of  sleep  deficit  and  fatigue.  [Source:  Costa, 

1999;  Warm  etal.,  1996] 

D  5.7.15  Vigilance  decrement.  The  effects  on  vigilance  due  to  the 
use  of  automation  should  be  considered  before  introducing  new 
automation.  [Source:  Warm  et  al.,  1996] 

Discussion.  A  vigilance  decrement,  that  is,  a 
continuously  decreasing  ability  to  maintain  attention  over 
time  while  monitoring,  may  occur  with  the  use  of 
automation. 

Vigilance  decrements  do  not  occur  because  monitoring 
tasks  are  under  stimulating.  Rather,  they  require  a  large 
amount  of  cognitive  resources  and  are  often  frustrating. 
Vigilance  decrements  have  been  observed  to  occur  for  both 
expert  and  novice  users  in  high  fidelity  simulations  and 
real-world  operations.  [Source:  Baker,  1962;  Colquhoun,  1967, 
1977;  Mackworth,  1948,  1961;  Schmidke,  1976  (as  found  in  Warm  et 
al.,  1996);  Warm  etal.,  1996] 

How  hard  the  user  must  work  in  order  to  maintain  vigilance 
can  be  determined  by  at  least  two  factors.  First,  workload 
is  affected  by  the  ease  with  which  relevant  signals  can  be 
detected.  Signals  that  have  low  salience  are  more  difficult 
to  detect  than  signals  high  in  salience.  Visual  fatigue  will 
also  require  more  effort  to  be  expended  in  order  to  detect  a 
signal.  Second,  musculo-skeletal  fatigue  associated  with 
maintaining  a  fixed  posture  will  increase  the  workload 
needed  to  perform  optimal  monitoring.  [Source:  Dember, 
Warm,  Nelson,  Simons,  Hancock,  &  Gluckman,  1993,  Warm  et  al., 
1996] 
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5.8  Fault  management 

Fault  management  relates  to  how  the  user  notices  and  recovers 
from  system  failures.  Such  failures  may  or  may  not  be  detected 
by  automation.  Fault  management  has  been  defined  to  include 
the  four  distinct  tasks  of  detection,  diagnosis,  prognosis,  and 
compensation.  [Source:  Rogers,  Schutte,  &  Latorella,  1996] 

■  5.8.1  Failure  recovery.  Automated  systems  shall  allow  for 
manual  control  and  preservation  of  safe  operations  should  the 
automation  or  one  or  more  components  of  the  system,  on  which 
the  automation  depends,  fail.  [Source:  NRC,  1998] 

Discussion.  The  resumption  of  manual  control  needs  to 
be  within  the  capacity  of  the  user,  without  relying  on 
manual  skills  that  may  be  degraded  by  the  use  of 
automation.  [Source:  NRC,  1998] 

■  5.8.2  Failures  made  apparent.  Automation  failures  shall  be 
made  unambiguously  obvious  to  the  user.  [Source:  AHCI,  1998; 
Billings,  1991] 

Discussion.  Stress,  preoccupation,  and  distraction  may 
reduce  the  user’s  ability  to  detect  faults.  [Source:  Rogers  et 
al.,  1996] 

D  5.8.3  Early  warning  notification.  Early  warning  notification  of 
pending  automation  failure  or  performance  decrements  should 
use  estimates  of  the  time  needed  for  the  user  to  adjust  to  task 
load  changes  due  to  automation  failure.  [Source:  Morrison, 
Gluckman,  &  Deaton,  1990] 

Discussion.  In  situations  where  automation  failure 
would  require  user  intervention,  it  is  useful  for  the  user 
to  be  warned  that  he  will  need  to  take  manual  control 
before  the  automated  system  fails.  Ideally,  this  warning 
needs  to  come  in  adequate  time  to  allow  the  user  to 
adjust  to  the  new  task  load.  There  may,  however,  be 
cases  where  it  is  not  possible  to  provide  advance 
notification  of  pending  failure  or  where  the  estimate  of 
time  needed  for  the  user  to  take  control  is  unknown. 
[Source:  Morrison  et  al.,  1990] 

■  5.8.4  Informing  user  of  potential  failure.  The  user  shall  be 
informed  of  automation  performance  decrements,  potential 
failures,  and  malfunctions.  [Source:  Billings,  1996] 

Discussion.  It  can  increase  workload  for  the  user  to 
continually  monitor  the  automation  for  failure.  Advance 
knowledge  about  potential  failures  can  also  help  the  user 
prepare  to  take  manual  control. 

■  5.8.5  Automated  diagnostic  aids.  Fault  isolation,  inspection, 
and  checkout  tasks  shall  be  automated  to  the  extent  practical. 
[Source:  National  Air  Space  Administration  (NASA-STD-3000A),  1989] 
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■  5.8.6  Automatic  self-checking  components.  All  essential 
electronic  computer  and  peripheral  components  that  are  part  of  a 
system  shall  incorporate  an  automatic  self-check  diagnostic  test  of 
software  and  hardware,  both  at  power  up  and  at  the  request  of  the 
operator,  to  ensure  they  are  functioning  properly.  [Source: 
Department  of  Defense,  (DoD  MIL-STD-1472D),  1981] 

■  5.8.7  On-demand  system  check.  On-demand  system  checkout 
shall  be  available.  [Source:  N AS A-STD-3000A ,  1989] 

■  5.8.8  Sensor  verification.  The  status  of  sensors  on  replacement 
units  shall  be  verifiable  with  respect  to  accuracy  and  proper 
operation.  [Source:  NASA-STD-3000A,  1989] 

■  5.8.9  Equipment  verification.  When  feasible,  equipment  shall 
permit  verification  of  operational  status  prior  to  installation 
without  the  need  for  disassembly.  [Source:  NASA-STD-3000A, 

1989] 

■  5.8.10  Fault  detection  without  disassembly.  Equipment  shall 
permit  fault  detection  and  isolation  without  removing 
components,  through  the  use  of  built-in  test,  integrated 
diagnostics,  or  standard  test  equipment.  [Source:  Department  of 
Defense  (MIL-STD-1800A),  1990;  NASA-STD-3000A,  1989] 

■  5.8.11  Rapid  fault  detection.  Equipment  design  shall  facilitate 
rapid  fault  detection  and  isolation  of  defective  items  to  permit 
their  prompt  removal  and  replacement.  [Source:  DoD  MIL-STD- 
1472D,  1981;  NASA-STD-3000A,  1989] 

■  5.8.12  Fault  detection  without  ambiguity.  Fault  detection  and 
isolation  shall  identify  without  ambiguity  which  component  has 
failed.  [Source:  DoD  MIL-STD-1800A,  1990;  NASA-STD-3000A,  1989] 

■  5.8.13  Portable  diagnostic  tools.  When  built-in  test  equipment  is 
not  available,  diagnostic  tools  or  portable  equipment  shall  be 
provided  to  aid  in  fault  isolation.  [Source:  NASA-STD-3000A,  1989] 

0  5.8.14  First-event  processing.  Automated  warning  systems 

should  provide  a  means  for  identifying  the  first  event  in  a  series 
of  alarm  events.  [Source:  NUREG-0700,  1996] 

Discussion.  When  a  series  of  interrelated  alarms  occur, 
information  identifying  which  component  first  exceeded 
the  set  threshold  can  be  valuable  in  determining  the 
initiating  cause  of  a  problem.  [Source:  NUREG-0700,  1996] 
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D  5.8.15  Diagnostic  information.  The  user  should  be  provided 
with  sufficient  information  and  controls  to  diagnose  automated 
warning  system  operation.  [Source:  Wiener  &  Curry,  1980] 

Discussion.  In  order  for  the  user  to  diagnose  the 
automated  system,  diagnostics  information  needs  to  be 
self-explanatory  and  in  plain  English.  The  diagnostic 
information  must  provide  the  user  with  the  information 
they  need  without  requiring  the  user  to  seek  additional 
references,  or  a  help  function,  to  understand  the  problem 
and  the  recommended  solution. 

5.9  False  alarms 


D  5.9.1  False  alarm  rates.  False  alarm  rates  should  not  be  so 
frequent  as  to  cause  the  user  to  mistrust  the  automated  system. 
[Source:  NUREG/CR-6105,  1994;  Parasuraman,  Hancock,  &  Olofinboba, 
1997;  Wiener  &  Curry,  1980] 

Discussion.  The  trade-off  between  alerting  the  user  to 
off-normal  conditions  and  the  creation  of  nuisance  alarms 
needs  to  be  considered  when  determining  appropriate 
alarm  set  points.  A  system  that  is  designed  to  minimize 
misses,  at  all  costs,  is  likely  to  have  frequent  false 
alarms.  However,  automated  systems  that  have  frequent 
false  alarms  are  unlikely  to  be  trusted  or  even  tolerated. 
When  there  is  low  probability  that  the  alarm  is  a  true 
alarm  (the  “cry-wolf  ’  phenomenon),  users  tend  to 
ignore,  mistrust  or  turn  off  alarms.  Setting  the  false 
alarm  threshold  requires  careful  evaluation  of  the  trade 
offs  between  missed  signals  and  false  alarms  including 
not  only  the  decision  thresholds  at  which  the  system  is 
set,  but  also  the  probabilities  of  the  condition  to  be 
detected.  [Source:  NRC,  1997] 

D  5.9.2  Inform  users  of  the  probability  of  a  true  alarm.  Users 
should  be  informed  of  the  inevitable  occurrence  of  automation 
false  alarms  particularly  when  base  rates  are  low.  [Source:  NRC, 
1998] 


Discussion.  When  the  probability  of  an  event  is  low,  the 
odds  of  a  true  alarm  can  be  quite  low  for  even  a  very 
sensitive  warning  system,  causing  inevitable  false 
alarms.  [Source:  NRC,  1998;  Parasuraman  et  al.,  1997] 
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5.10  Training 


D  5.10.1  Introducing  new  automation.  New  automation  should 
be  introduced  with  advanced  briefing  and  subsequent  training 

procedures.  [Source:  Billings,  1997;  NRC,  1998;  Parasuraman  &  Riley, 
1997]  J 


Discussion.  The  introduction  of  new  automation  may 
introduce  changes  in  traditional  roles  and  responsibilities, 
a  redistribution  of  authority  for  tasks  or  changes  to  the 
nature  of  the  cognitive  demands  imposed  on  the  human 
operator.  [Source:  Bowers  et  al . ,  1995;  Wiener,  1989] 

D  5.10.2  Prepare  users  for  changes.  Before  automation  is 

introduced,  users  should  be  informed  of  associated  changes  and 
increases  in  the  work  effort,  as  well  as  the  benefits  associated 
with  the  automation.  [Source:  DISA,  1996;  Scerbo,  1996] 

Discussion.  The  roles  and  responsibilities  of  the  users, 
cognitive  demands,  and  operational  procedures  may 
change  as  a  result  of  introducing  automation.  [Source: 
Bowers,  Deaton,  Oser,  Prince,  &  Kolb,  1995] 


D  5.10.3  Understanding  automated  functions.  Initial  training  in 
the  use  of  automation  should  be  sufficient  for  the  users  to  fully 
understand  how  the  automation  functions  within  the  particular 

system,  as  well  as  how  to  use  the  automation.  [Source:  Billings, 
1997]  6 


Discussion.  Lack  of  knowledge  and  understanding  of 
how  automation  works  can  make  it  difficult  for  users  to 
assess  potential  problems  and  may  result  in  improper  use 
of  automation.  [Source:  Rudisill,  1995] 

°  5.10.4  Backup  training.  Users  should  be  provided  with  backup 

training  in  performing  any  tasks  replaced  by  automation  or  in 

operating  any  backup  systems  replaced  by  automation.  [Source- 
DISA,  1996] 
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n  5.10.5  Inappropriate  use  of  automation.  Users  should  be 
trained  to  recognize  inappropriate  uses  of  an  automated  tool 
including  automation  bias  (the  use  of  automation  in  a  heuristic 
manner  as  opposed  to  actively  seeking  and  processing 
information).  [Source:  DISA,  1996;  Dzindolet,  Pierce,  Beck,  &  Dawe, 
1999;  Mosier  &  Skitka,  1999] 

Discussion.  There  are  different  categories  of 
inappropriate  automation  use,  including  automation  bias, 
ignoring  or  turning  off  the  automation,  and  improper 
implementation  of  automation. 

Users  may  rely  on  automated  decision  aids  in  a  heuristic 
manner  (referred  to  as  automation  bias).  Using 
heuristics  is  to  apply  simple  decision-making  rules  to 
make  inferences  or  to  draw  conclusions  simply  and 
quickly.  Heuristics  are  useful  principles  having  wide 
application,  but  may  not  be  strictly  accurate.  Usually  a 
heuristic  strategy  is  optimal,  however,  under  certain 
conditions  heuristics  will  be  inappropriate  and  errors  or 
misuse  may  occur.  Automation  bias  leads  to  errors  of 
omission  (failure  to  notice  system  anomalies  when 
automation  fails)  and  errors  of  commission  (acceptance 
of  automated  decisions  without  cross-checking  or  in 
presence  of  contradictory  information).  Training  will 
help  prevent  automation  bias  and  help  the  user  learn  to 
examine  multiple  sources  of  information  before  making  a 
decision.  Early  training  on  automation  bias  may  reduce 
commission  errors  for  users  new  to  automation,  but  may 
be  less  likely  to  reduce  omission  errors  or  errors  made 
by  expert  users. 

Inappropriate  use  of  automation  may  be  influenced  by 
various  individual  factors  such  as  self-confidence  in 
completing  the  task,  trust  in  the  automation,  differential 
effects  of  fatigue,  and  how  all  of  these  factors  combined 
weigh  into  the  decision  making  process.  Inappropriate 
use  of  automation  can  be  due  to  misuse  (automation  bias, 
complacency),  disuse  (ignoring  or  turning  off 
automation)  or  abuse  (improper  implementation  of 
automation).  [Source:  Dzindolet  et  al.,  1999;  Lee  &  Moray, 
1992;  Mosier  &  Skitka,  1996;  Mosier,  Skitka,  Dunbar,  Burdick, 
McDonnell,  &  Rosenblatt,  1998;  Muir,  1987  (as  found  in  Scerbo, 
1996);  Parasuraman  &  Riley,  1997;  Riley,  1996] 
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5.10.6  Automation  reliability  training.  Users  should  be  trained 
to  recognize  and  understand  the  conditions  under  which 
automation  may  be  unreliable,  and  to  learn  the  conditions  where 
it  performs  well  (when  or  when  not  to  question  the  automation). 
[Source:  Cohen  et  al.,  1998;  Dzindolet  et  al.,  1999] 

Discussion.  Users  must  learn  not  to  categorically  accept 
the  recommendation  of  a  decision  aid.  Understanding 
the  automation’s  weaknesses  allows  users  to  better  judge 
how  much  they  should  trust  the  automation  without 
becoming  overconfident  in  its  performance.  This 
recognition  process  may  impose  an  additional  workload 
on  the  user.  [Source:  Dzindolet  et  al.,  1999] 

5.10.7  Over-reliance  on  automation.  Users  should  be  trained 
not  to  become  overly  reliant  on  automation.  [Source:  Mosier, 
Skitka,  Heers,  &  Burdick,  1997;  Parasuraman  &  Riley,  1997] 

Discussion.  When  users  rely  on  automation  too  much 
they  become  susceptible  to  automation-induced 
complacency.  Monitoring  failures  are  likely  to  occur 
when  users  become  overly  reliant  on  automation.  [Source: 
Mosier  et  al.,  1994;  Parasuraman  et  al.,  1993] 

5.10.8  Risk  assessment  and  reduction.  Users  should  be  trained 
on  risk  assessment  and  actions  needed  for  risk  reduction. 

[Source:  Mosier  &  Skitka,  1999] 

5.10.9  Transition  training.  Users  should  be  trained  on 
transitioning  from  automated  to  conventional  systems.  [Source: 
Rudisill,  1994] 

Discussion.  If  automation  were  to  fail,  users  need  to  be 
skilled  at  both  recognizing  the  failure  and  taking  manual 
control . 

5.10.10  User-automation  interaction.  Training  programs 
should  stress  user-automation  interaction  skills  and 
cognitive/problem  solving  skills  rather  than  psychomotor  skills. 
[Source:  Sarter  &  Woods,  1994] 

Discussion.  Problems  in  automation  may  not  be  inherent 
in  the  technology  itself.  Problems  can  arise  due  to 
limitations  in  the  integration  of  the  user  and  automation. 
The  user  and  automation  should  be  integrated  by 
developing  a  joint,  distributed  cognitive  system  by  means 
of  training  and  design.  [Source:  Sarter  &  Woods,  1994] 

5.10.11  Training  for  changes  due  to  automation.  When 
automation  requires  different  kinds  of  cognitive  processing, 
ways  of  thinking,  and  discarding  of  traditional  methods  and 
skills,  then  training  should  be  designed  to  address  problems 
related  to  these  changes.  [Source:  Garland  &  Hopkin,  1994] 
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D  5.10.12  Automation  output.  Users  should  be  trained  on  what 
constitutes  the  normal  automation  output  so  that  the  user  can 
easily  determine  whether  the  system  is  functioning  properly. 
[Source:  Morris  et  al.,  1985] 


5.11  Function  allocation/levels  of  automation 

There  are  many  possible  levels  of  automation  (see  Table  1) 
including:  automation  that  automatically  executes  tasks, 
automation  that  performs  tasks  when  pre-specified  conditions 
are  met,  and  automation  that  suggest  a  course  of  action  or 
facilitates  a  decision.  [Source:  Billings,  1997;  NRC,  1998;  Parasuraman 
et  al.,  2000] 

Table  1.  Levels  of  automation,  from  high  to  low.  [Source:  NRC, 
1998;  Sheridan,  1996] 


The  system  acts  autonomously  without  human  intervention 
The  system  informs  the  user  after  executing  the  action 

only  if  the  system  decides  it  is  necessary _ 

The  system  informs  the  user  after  executing  the  action 

only  upon  user  request _ 

The  system  executes  an  action  and  then  informs  the  user 
The  system  allows  the  user  a  limited  time  to  veto  before 

executing  an  action _ _ 

The  system  executes  an  action  upon  user  approval _ 

The  system  suggests  one  alternative _ 

The  system  narrows  the  selection  down  to  a  few _ 

The  system  offers  a  complete  set  of  action  alternatives 
The  system  offers  no  assistance _ 


0  5.11.1  Function  allocation  alternatives.  Alternative  function 

allocations  including  fully  manual,  partially  automated,  fully 
automated,  and  adaptive  allocation  should  be  evaluated  for 
feasibility  and  effectiveness.  [Source:  Wiener  &  Curry,  1980] 

D  5.11.2  Evaluated  through  simulation.  Alternative  schemes  for 
the  allocation  of  functions  should  be  examined  in  the  context  of 
the  whole  system  through  the  use  of  high  fidelity  simulations. 
[Source:  Wiener  &  Curry,  1980] 

Discussion.  Because  there  may  be  multiple  potential 
schemes  in  the  allocation  of  functions,  simulating  these 
schemes  in  the  context  of  the  whole  system  will  allow 
them  to  be  evaluated  properly.  A  scheme  that  seems  to 
be  the  most  appropriate  in  regards  to  accomplishing  a 
specific  task  may  not  be  the  best  choice  in  relation  to  the 
functioning  of  the  entire  automated  system. 

D  5.11.4  Functions  to  automate.  Only  functions  that  are 

performed  well  by  machines  should  be  automated,  not  functions 
that  are  performed  better  by  humans.  [Source:  Drury,  1998] 
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D  5.11.5  Automate  full  behavioral  modules.  Behavioral  modules 
in  their  entirety  should  either  be  automated  or  preserved  as 
manual  subtasks,  not  fractionally  (partially)  automated.  [Source: 
Vortac,  Barile,  Albright,  Truitt,  Manning,  &  Bain,  1996] 

Discussion.  A  behavioral  module  is  a  unitized  set  of 
actions  that  can  be  performed  in  an  over-learned, 
automatic  fashion  with  very  little  effort.  When  a  set  of 
cognitive  or  behavioral  actions  is  frequently  performed 
together  they  will  eventually  form  a  module. 

Automation  that  replaces  only  a  portion  of  a  module  will 
produce  no  advantage  in  performance  and  may  inhibit 
performance.  [Source:  Vortac  et  al.,  1996] 

D  5.11.6  User  tasks.  Tasks  that  are  performed  in  an  unpredictable 
environment  requiring  flexibility  and  adaptability  should  be 
allocated  to  the  user.  [Source:  AHCI,  1998] 

0  5.11.7  Clear  roles  and  responsibilities.  The  automated  system 

should  make  it  clear  whether  the  user  or  computer  is  supposed 
to  perform  a  particular  task  at  a  specific  time.  [Source: 
Parasuraman  &  Riley,  1997] 

D  5.11.8  Changing  roles  and  responsibilities.  The  automated 
system  should  provide  a  means  for  changing  the  allocation  of 
roles  and  responsibilities.  [Source:  Parasuraman  &  Riley,  1997] 

°  5.11.9  Automation  of  high-risk  actions  or  decisions.  For 

system  tasks  associated  with  greater  uncertainty  and  risk, 
automation  should  not  proceed  beyond  the  level  of  suggesting  a 
preferred  decision/action  alternative.  [Source:  NRC,  1998] 


Discussion.  High  levels  of  automation  can  be  used  for 
tasks  involving  relatively  little  uncertainty  and  risk. 
[Source:  NRC,  1998] 


5.12  Information  automation 

Information  automation  includes  information  acquisition  and 
integration.  This  type  of  automation  would  include  filtering, 
distributing  or  transforming  data,  providing  confidence  estimates 
and  integrity  checks,  and  enabling  user  requests. 

D  5.12.1  Incomplete,  missing,  uncertain,  and  invalid  data.  The 
automated  system  should  provide  a  means  to  indicate  to  the  user 
that  data  are  missing,  incomplete,  unreliable,  or  invalid  or  that 
the  system  is  relying  on  backup  data.  [Source:  AHCI,  1998] 

D  5.12.2  Automatic  update.  When  the  displayed  data  are  changed 
as  a  result  of  external  events,  the  user  should  be  provided  with 
the  option  of  having  an  automatic  update  of  changed 
information.  [Source:  AHCI,  1998] 
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D  5.12.3  Multiple  formats.  System  designers  should  provide 
information  in  multiple  formats  (e.g.,  text,  graphics,  voice,  and 
video)  to  allow  better  communication  and  reduction  of 
workload.  [Source:  Scerbo,  1996] 

Discussion.  Communication  will  be  improved  by 
allowing  information  to  be  presented  in  the  most 
understandable  format.  Eliminating  the  need  to  translate 
information  into  a  specific  format  will  reduce  workload. 
[Source:  Scerbo,  1996] 

D  5.12.4  Accurate  reflection  of  status.  Information  presented  to 
the  user  should  accurately  reflect  system  and  environment  status 
in  a  manner  so  that  the  user  rapidly  recognizes,  easily 
understands,  and  easily  projects  system  outcomes  in  relation  to 
system  and  user  goals.  [Source:  Endsley  &  Kiris,  1995;  NUREG-700, 
1996] 

°  5.12.5  Error  Mitigation.  Error-prone  conditions  should  be 

minimized  by  maintaining  user  awareness,  providing  adequate 
training,  developing  standard  operating  procedures,  and 
fostering  crew  coordination.  [Source:  Sheehan,  1995] 

Discussion.  Errors  due  to  automation  may  arise  from  data 
entry  errors,  monitoring  failures,  system  workarounds,  and 
mode  misapplication.  Error-prone  conditions  in  automated 
systems  may  result  from  lack  of  mode  awareness,  lack  of 
situation  awareness,  lack  of  systems  awareness,  increased 
heads  down  time,  over-dependence  on  automation,  and 
interrupted  crew  coordination.  Automation-related  errors 
usually  occur  in  conjunction  with  other  factors  such  as 
haste,  inattention,  fatigue,  distraction,  or  other  system 
factors.  [Source:  Sheehan,  1995] 

■  5.12.6  Information  display.  Information  displays  shall  support 
and  reinforce  status  and  situation  awareness  at  all  times.  [Source: 
Billings,  1991,  1996] 

Discussion.  A  primary  objective  of  information 
automation  is  to  maintain  and  enhance  situation 
awareness.  However,  too  much  information  presented 
simultaneously  may  become  cluttered  and  make  visual 
search  difficult,  interfering  with  status,  decision-making, 
or  control.  It  is  important  for  the  user  to  be  able  to 
easily  locate  needed  information.  [Source:  Billings,  1991] 

The  user’ s  ability  to  detect  a  signal  while  monitoring 
varies  inversely  with  the  rate  at  which  neutral 
background  events  are  repeated.  [Source:  Lanzetta,  Dember, 
Warm,  &  Berch,  1987;  Parasuraman,  1979  (as  found  in  Warm  et 
al.,  1996)] 

■  5.12.7  Situation  displays.  Event  data  should  be  combined  with 
a  map  background  when  the  geographic  location  of  changing 
events  needs  to  be  shown.  [Source:  Smith  &  Mosier,  1986] 
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5.12.8  Information  presentation.  Both  the  content  of  the 
information  made  available  through  automation  and  the  ways  in 
which  it  is  presented  shall  be  consistent  with  the  task  priorities. 
[Source:  Billings,  1996] 

5.12.9  Cueing  important  information.  When  information  must 
be  updated  quickly,  the  most  important  information  should  be 
cued  to  ensure  it  will  be  the  first  to  be  processed  by  the  user. 
[Source:  Wickens,  2000] 

Discussion.  It  is  important  that  the  cues  be  correct,  as 
there  may  be  significant  costs  of  invalid  cueing.  [Source: 
Wickens,  2000] 

5.12.10  Automatic  information  queuing.  Incoming  messages 
should  be  queued  automatically  by  the  system  so  they  do  not 
disrupt  current  information  handling  tasks.  [Source:  Smith  & 
Mosier,  1986] 

5.12.11  Highlighting  changed  data.  Data  changes  that  occur 
following  automatic  display  update  should  be  temporarily 
highlighted.  [Source:  Smith  &  Mosier,  1986] 

5.12.12  Lists  of  information.  Long  lists  of  information,  tasks, 
and  so  on,  should  be  stored  and  prioritized  by  the  automated  aid 
to  minimize  the  number  of  decision  alternatives  and  reduce  the 
visual  processing  load  of  human  operators.  [Source:  Barnes,  1981] 

5.12.13  Integration  of  display  elements.  Display  elements 
should  only  be  integrated  if  it  will  enhance  status  interpretation, 
decision-making,  situation  awareness,  or  other  aspects  of  task 
performance.  [Source:  Billings,  1991] 

5.12.14  Integrated  displays.  Integrated  displays  should  combine 
various  information  automated  system  elements  into  a  single 
representation.  [Source:  Billings,  1996;  Parasuraman  et  ah,  2000] 

Discussion.  Feedback  information  that  is  widely 
distributed  among  various  indicators  can  result  in 
insufficient  monitoring  of  automation  and/or  mode 
confusion.  In  such  cases,  monitoring  adequacy  is  limited 
by  inefficient  scanning  patterns  and  information  that  is 
difficult  to  integrate.  [Source:  Mosier  &  Skitka,  1999] 
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D  5.12.15  Integrated  or  non-integrated  information 

arrangement.  System  information  should  be  automatically 
reorganized  into  integrated  or  non-integrated  arrangements 
depending  on  the  current  system  status.  [Source:  Forester,  1987; 
Parasuraman,  et  al.,  1996] 

Discussion.  Integrated  information  arrangement  allows 
the  user  to  assess  the  overall  status  of  the  system. 
Integrating  display  components  into  aggregated 
arrangements  may  reduce  the  attention  demands  of  fault 
detection.  Non-integrated  arrangement  of  components 
draws  user  attention  to  system  errors  or  other  relevant 
information.  Presenting  the  information  in  a  format 
relevant  to  the  state  of  the  system  can  facilitate  the  ability 
of  the  user  to  quickly  and  easily  assess  the  system  status. 
[Source:  Forester,  1987;  Parasuraman  et  al.,  1996] 

a  5.12.16  Equally  prominent  cues.  Automated  and  non-automated 
cues  should  be  made  equally  prominent  to  enable  users  to  collect 
confirming/disconfirming  evidence  before  deciding  on 
appropriate  action.  [Source:  Mosier  &  Skitka,  1999] 

Discussion.  Automation  bias,  the  tendency  to  use 
automation  in  a  heuristic  manner,  may  be  suppressed  if 
other,  non-automated  sources  of  information  are 
presented  with  salience  equal  to  that  of  the  automated 
information.  [Source:  Mosier  &  Skitka,  1999] 


5.13  Adaptive  automation 

Adaptive  automation  is  the  real  time  allocation  of  tasks  to  the 
user  or  automated  system  in  a  flexible  manner,  changing  the 
automation  to  meet  current  situational  demands.  Adaptive 
automation  may  benefit  user  performance  by  allowing  the  user 
to  remain  in  active  control  of  the  system  instead  of  becoming  a 
passive  observer.  Active  control  may  prevent  performance 
decrements  associated  with  long-term  monitoring,  loss  of 
situation  awareness  and  manual  skill  degradation.  [Source: 
Morrison  et  al.,  1990;  NRC,  1998;  Scerbo,  1996;  Scerbo  &  Mouloua,  1999] 

Discussion.  Laboratory  experiments  have  shown  that 
short  periods  of  automation  use  (e.g.,  10-minute  cycles 
of  manual  and  automated  control)  do  not  result  in 
performance  decrements.  This  suggests  that  intermittent 
periods  of  manual  control  may  help  to  maintain 
.  performance  in  the  presence  of  automation.  [Source: 
Gluckman,  Carmody,  Morrison,  Hitchcock,  &  Warm,  1993  (as 
found  in  Scerbo,  1996);  Parasuraman  et  al.,  1991] 
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D  5.13.1  Help  during  high  workload.  Automation  should  be 
designed  to  adapt  by  providing  the  most  help  during  times  of 
highest  user  workload,  and  somewhat  less  help  during  times  of 
lowest  workload.  [Source:  Billings,  1996;  Parasuraman,  Mouloua,  & 
Hilburn,  1998] 

Discussion.  Research  has  shown  that  adaptive 
automation  may  reduce  mental  workload  most  effectively 
during  periods  of  high  taskload.  [Source:  Hilburn  et  al., 
1996] 

D  5.13.2  Adaptive  automation  timing.  Adaptive  automation 
should  not  be  implemented  unexpectedly  or  at  a  time  when  the 
user  may  not  desire  the  aiding.  [Source:  Scerbo,  1996] 

Discussion.  The  timing  of  adaptation  may  have  critical 
impact  on  user  acceptance  of  automation.  Studies  show 
that  users  prefer  to  be  in  control  of  the  system.  However, 
there  are  times  that  automation  may  need  to  be  initiated  by 
the  system,  particularly  when  changes  in  workload  occur 
rapidly  or  are  unexpected  by  the  user.  [Source:  Harris, 
Goemert,  Hancock,  &  Arthur,  1994  (as  found  in  Scerbo,  1996)] 

D  5.13.3  Adaptive  automation  implementation.  Adaptive 

automation  should  be  implemented  at  the  point  at  which  the  user 
ignores  a  critical  amount  of  information.  [Source:  Sen,  1984] 

Discussion.  Fatigue  (or  other  factors)  may  prevent  users 
from  recognizing  the  best  time  to  utilize  automation  and 
performance  decrements  may  consequently  occur.  One 
indication  that  the  user  is  being  overloaded  is  an  increase 
in  the  amount  of  information  he  must  ignore  in  order  to 
make  a  timely  decision.  Thus,  the  designer  can  use  a 
threshold  critical  amount  of  ignored  information  as  an 
indicator  that  the  user  is  overloaded  and  implement 
adaptive  automation  at  that  point  (to  help  reduce 
workload).  What  constitutes  a  critical  amount  of 
information  can  vary  depending  on  the  particular  task 
and  may  best  be  determined  on  a  system-by-system  basis. 
[Source:  Harris,  Hancock,  &  Arthur,  1993  (as  found  in  Scerbo, 
1996);  Sen,  1984] 

D  5.13.4  Adapting  to  skill  of  the  user.  Adaptive  automation 
should  be  used  to  increase  the  performance  of  users  with 
different  skill  levels.  [Source:  Norico  &  Stanley,  1989] 

Discussion.  By  adapting  to  the  skill  of  the  user,  adaptive 
automation  can  increase  the  proficiency  of  the  novice 
user  and  prevent  frustration  that  might  otherwise  occur 
with  complex  systems. 

D  5.13.5  Skill  of  adaptive  automation.  Adaptive  automation 
should  be  at  least  as  skilled  as  the  user,  if  not  greater,  to 
promote  optimal  user  performance.  [Source:  Woods,  1996] 
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D  5.13.6  Modeling  of  human  behavior.  Modeling  of  human 
behavior  for  aid-initiated  intervention  should  at  least  include: 
task  execution  goal  states,  environment  representation 
(graphical),  situation  assessment  information  and  planning,  and 
commitment  logic.  [Source:  Andes  &  Hunt,  1989] 

Discussion.  When  modeling  user  behavior,  it  ought  to 
be  noted  that  users  vary  greatly  in  the  way  they  employ 
automation.  [Source:  Lee,  1992;  Lee  &  Moray,  1992  (as  found  in 
Riley,  1996)] 


D  5.13.7  Interface  adaptation.  When  dynamic  adaptation  of  the 
interface  is  used,  it  should  be  attained  by  utilizing  information 
provided  to  the  system  through  user  interactions  within  a 
specific  context.  [Source:  Norico  &  Stanley,  1989] 

Discussion.  Dynamic  adaptation  of  the  interface  may 
promote  operator  acceptance  of  automation. 


D  5.13.8  Menu  adaptation.  When  dynamic  adaptation  of  menus  is 
used,  the  resultant  menus  should  offer  only  the  options  that  are 
relevant  to  the  current  environment.  [Source:  Barnes,  1985] 


Discussion.  Dynamic  adaptation  of  the  menus  occurs 
when  menus  are  altered  to  reflect  the  needs  of  the  current 
environment.  This  approach  may  reduce  user  workload. 
[Source:  Barnes,  1985] 

D  5.13.9  Direct  manipulation  interface.  Direct  manipulation 
interfaces  should  be  used  to  minimize  the  impact  of  a  transition 
to  manual  control.  [Source:  Morrison  et  al.,  1993] 

Discussion.  An  example  of  direct  manipulation  is  a 
graphical  user  interface  (GUI).  In  direct  manipulation, 
the  user  controls  the  interaction  with  the  computer  by 
acting  directly  on  objects  on  the  display  screen.  An 
object  may  be  an  icon,  menu  option,  symbol,  button,  or 
dialog  box.  (See  HFDG  Update:  A  Revision  to  Chapter 
8  (Ahlstrom  &  Longo,  2001)  on  computer-human  interfaces 
for  more  information  on  direct  manipulation.)  [Source: 
Shneiderman,  1992] 
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5.14  Decision  aids 


Definition.  Decision  aids  (sometimes  referred  to  as  decision 
support  systems)  are  automated  systems  that  provide  support  to 
human  decision-making  processes  either  unsolicited  or  by  user 
request.  Decision  aids  can  narrow  the  decision  alternatives  to  a 
few  or  suggest  a  preferred  decision  based  on  available  data. 
[Source:  Wiener,  1988] 

D  5.14.1  Use.  Decision  aids  should  be  used 

a.  for  managing  system  complexity; 

b.  for  assisting  users  in  coping  with  information  overload; 

c.  for  focusing  the  user’s  attention; 

d.  for  assisting  the  user  in  accomplishing  time-consuming 
activities  more  quickly; 

e.  when  limited  data  results  in  uncertainty; 

f.  for  overcoming  human  limitations  that  are  associated 
with  uncertainty,  the  emotional  components  of  decision¬ 
making,  finite-memory  capacity,  and  systematic  and 
cognitive  biases;  and 

g.  for  assisting  the  user  in  retrieving,  retaining, 
representing  or  manipulating  large  amounts  of 
information,  combining  multiple  cues  or  criteria, 
allocating  resources,  managing  detailed  information, 
performing  computations,  and  selecting  and  deciding 
among  alternatives.  [Source:  AHCI,  1998;  DISA,  1996] 


0  5.14.2  When  to  avoid.  Decision  aids  should  not  be  used 

a.  when  solutions  are  obvious; 

b.  when  one  alternative  clearly  dominates  all  other  options; 

c.  when  there  is  insufficient  time  to  act  upon  a  decision; 

d.  when  the  user  is  not  authorized  to  make  decisions;  or 

e.  for  cognitive  tasks  in  which  humans  excel,  including 
generalization  and  adapting  to  novel  situations.  [Source: 
AHCI,  1998] 


D  5.14.3  User  determination  of  decision  aid  use.  Users  should  be 
able  to  determine  when  and  how  the  decision  aid  should  be 
used.  [Source:  Parasuraman  &  Riley,  1997] 
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°  5.14.4  Appropriate  to  user.  Decision  aids  should  use 

terminology  and  criteria  appropriate  to  the  target  user  group. 
[Source:  DISA,  1996] 

a  5.14.5  Reduce  number  of  response  options.  Decision  aids 
should  reduce  the  number  of  response  options.  [Source:  Barnes, 
1985] 

Discussion.  The  number  of  options  that  the  user  must 
consider  is  expected  to  decrease  when  a  decision  aid  is 
used.  Reducing  the  response  options  focuses  the  user’s 
attention  onto  the  most  viable  options. 

D  5.14.6  Assist  user  decisions.  Decision  aids  should  assist,  rather 
than  replace,  human  decision  makers  by  providing  data  for 
making  judgments  rather  than  commands  that  the  user  must 
execute.  [Source:  AHCI,  1998;  DISA,  1996;  Parasuraman  &  Riley,  1997] 

D  5.14.7  Consistent  with  mental  models.  The  support  provided 
by  decision  aids  should  be  consistent  with  user  cognitive 
strategies  and  expectations  (mental  models).  [Source:  NUREG-700, 
1996] 

Definition.  A  mental  model  is  an  individual’s 
understanding  of  the  processes  underlying  system 
operation.  [Source:  NRC,  1998;  Parasuraman  et  al.,  1996] 

0  5.14.8  Non-cancellation  of  ongoing  tasks.  Use  of  decision  aids 

should  not  require  on-going  user  tasks  to  be  cancelled.  [Source: 
NUREG-700,  1996] 

D  5.14.9  Minimize  query  of  user.  Decision  aids  should  minimize 
query  of  the  users  for  information.  [Source:  NUREG-0700,  1996] 

D  5.14.10  Minimize  data  entry.  Decision  aids  should  minimize 
user  data  entry  requirements.  [Source:  DISA,  1996] 

D  5.14.11  Planning  strategy  or  guiding  process.  Decision  aids 
should  be  capable  of  planning  a  strategy  to  address  a  problem  or 
guide  a  complex  process.  [Source:  NUREG-0700,  1996] 

D  5.14.12  Accept  user  direction.  Decision  aids  should  accept 
direction  from  the  users  on  which  problem  solving  strategy  to 
employ  when  alternative  strategies  are  available.  [Source:  NUREG- 
0700,  1996] 

D  5.14.13  Prioritize  alternatives.  When  more  than  one  alternative 
is  available,  the  decision  aid  should  provide  the  alternatives  in  a 
recommended  prioritization  scheme  based  on  mission  and  task 
analysis.  [Source:  AHCI,  1998] 

0  5.14.14  Unable  to  process.  Decision  aids  should  alert  the  user 

when  a  problem  or  situation  is  beyond  its  capability.  [Source: 
NUREG-0700,  1996] 
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5.14.15  Type  and  sequence  of  input.  Decision  aids  should  be 
flexible  in  the  types  and  sequencing  of  user  inputs  accepted. 
[Source:  NUREG-0700,  1996] 

5.14.16  Uncertainty  estimate  and  rationale.  Decision  aids 
should  estimate  and  indicate  the  certainty  of  analysis  and  provide 
the  rationale  for  the  estimate.  [Source:  NUREG-0700,  1996] 

5.14.17  Derived  or  processed  data.  When  information  used  by 
a  decision  aid  is  derived  or  processed,  the  data  from  which  it  is 

derived  should  be  either  visible  or  accessible  for  verification. 
[Source:  Billings,  1996] 

Discussion.  Data  that  are  not  critical  for  operation  can 
be  made  available  only  upon  request. 

5.14.18  Hard  copy  of  decision  aid  use.  The  user  should  be  able 
to  obtain  hard  copy  print  outs  of  data  including  screen  displays, 
rules  and  facts,  data  employed,  hypotheses  tested,  and  summary 
information.  [Source:  NUREG-0700,  1996] 

5.14.19  Access  to  procedural  information.  Decision  aids 
should  give  the  user  access  to  procedural  information  used  by 
the  aid.  [Source:  Morris,  Rouse  &  Ward,  1985;  NUREG-0700,  1996] 

Discussion.  Procedural  information  is  information 
about  the  rules  or  algorithms  used  by  the  decision  aid. 
Knowledge  of  procedural  information  fosters  user 
acceptance  of  the  aid  because  the  user  is  able  to 
understand  how  the  aid  functions.  As  the  user  becomes 
more  familiar  with  a  given  situation,  he  requires  less 
procedural  information.  [Source:  Morris  et  al.,  1985] 

5.14.20  Explanation  detail.  When  the  system  provides 
explanations  to  the  user,  it  should  supply  a  short  explanation 
initially,  with  the  ability  to  make  available  more  detail  at  the 
user’s  request,  including  access  to  process  information  or  an 
explanation  for  the  rules,  knowledge-basis,  and  solutions  used 
by  the  decision  aid.  [Source:  DISA,  1996;  NUREG-0700,  1996] 

Discussion.  Process  information  is  the  information 
about  how  the  aid  accomplishes  a  task.  This  information 
is  required  by  users  to  decide  whether  to  use  the  aid  in 
unfamiliar  situations  and  for  identifying  the  nature  and 
extent  of  malfunctions.  [Source:  Morris  et  al.,  1985] 

5.14.21  Clear  explanations  to  user.  When  the  system  provides 
explanations  to  the  user,  the  explanation  should  use  terms 
familiar  to  the  user  and  maintain  consistency  with  the  immediate 
task.  [Source:  DISA,  1996] 

5.14.22  Information  presentation.  Decision  aids  should  present 
information  at  the  level  of  detail  that  is  appropriate  to  the 

immediate  task,  with  no  more  information  than  is  essential. 
[Source:  AHC1,  1998] 
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5.14.23  Avoid  repeated  information.  Decision  aids  should 
avoid  repeating  information  that  is  already  available.  [Source: 
AHCI,  1998] 

5.14.24  Decision  aid  integration.  Decision  aids  should  be  fully 
integrated  and  consistent  with  the  rest  of  the  computer-human 
interface.  [Source:  NUREG-0700,  1996] 

5.14.25  Alert  to  newly  available  information.  Decision  aids 
should  alert  the  user  to  changes  in  the  status  of  important  system 
information  such  as  when  critical  information  becomes  available 
during  decision  aid  utilization.  [Source:  NUREG-0700,  1996] 

Discussion.  Critical  information  in  this  guideline  refers 
to  information  that  may  have  a  significant  impact  on  task 
completion. 

5.14.26  Alert  to  meaningful  events  or  patterns.  Decision  aids 
should  automatically  notify  the  user  of  meaningful  patterns  or 
events  such  as  when  it  predicts  a  future  problem.  [Source:  AHCI, 
1998] 

5.15.27  Predict  based  on  historical  data.  Decision  aids  should 
be  able  to  predict  future  data  based  on  historical  data  and 
current  conditions.  [Source:  AHCI,  1998] 

5.14.28  Graphic  representation.  Decision  aids  should  be  able 
to  graphically  represent  system  relationships,  its  rules  network, 
and  reasoning  process.  [Source:  NUREG-0700,  1996] 

5.14.29  Simulation  mode  identification.  When  decision  aids 
have  a  simulation  mode,  entering  the  simulation  mode  should 
require  an  explicit  command  and  result  in  a  distinguishable 
change  in  output.  [Source:  NUREG-0700,  1996] 

5.14.30  Knowledge  of  intent.  Each  element  in  an  intelligent 
human-machine  system  shall  have  knowledge  of  the  intent  of  the 
other  elements.  [Source:  Billings,  1996;  NRC,  1998;  Parasuraman  et  al., 
2000] 

Discussion.  Monitoring  of  the  system  by  the  user  and 
the  user  by  the  system  can  only  be  effective  if  each 
knows  what  the  other  one  is  trying  to  accomplish. 

[Source:  Billings,  1996] 

5.14.31  Adaptive  decision  aiding.  When  adaptive  decision 
aiding  is  used,  the  level  of  decision  aiding  should  change  with 
the  situational  demands  in  order  to  optimize  performance  (See 
section  5.13  on  adaptive  automation).  [Source:  Rouse,  1988] 

Discussion.  The  criticality  of  a  given  task  can  change 
dramatically  depending  on  the  current  situation.  [Source: 
Derrick,  1988] 
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D  5.14.32  Adaptive  decision  aiding  implementation.  Adaptive 
decision  aiding  should  be  applied  when  resource  loading, 
performance,  error  frequency,  and  deviations  from  intent  exceed 
threshold  levels  (See  section  5.13  on  adaptive  automation). 
[Source:  Andes,  1987] 


Discussion.  Resource  loading,  performance,  errors,  and 
deviations  from  intent  can  be  used  as  indicators  to 
determine  when  the  user  might  need  the  help  of  the 
automated  decision  aid.  The  threshold  levels  of  these 
indicators,  specifying  the  optimal  time  to  implement 
decision  aiding  may  need  to  be  determined  on  a  system- 
by-system  basis,  possibly  through  simulation. 


D  5.14.33  Planning  assistance.  Adaptive  decision  aiding  interfaces 
should  allow  the  user  to  receive  direct  assistance  in  planning 
how  to  carry  out  the  intended  task.  [Source:  Tyler  &  Treu,  1989] 

D  5.14.34  User  initiated  automation  implementation.  The  user 
should  be  able  to  initiate  automated  aids  even  if  system-initiated 
automation  is  the  norm.  [Source:  Billings,  1997;  Harris,  Hancock, 
Arthur,  &  Caird,  1995] 


Discussion.  User  acceptance  of  automation  centers  on 
whether  the  user  feels  in  control  of  the  system.  [Source: 
Rouse,  1988] 


5.15  Control  automation 

Definition.  Control  automation  is  when  the  system  executes 
actions  or  control  tasks  with  some  level  of  autonomy. 

°  5.15.1  Actions  similar  to  human  control.  When  automated 

control  actions  are  performed,  the  automated  tasks  should  be 
easily  understood  by  users  and  similar  to  user  control  actions. 
[Source:  Billings,  1991] 

D  5.15.2  Authority  limits.  Control  automation  should  not  be  able 
to  jeopardize  safety  or  make  a  difficult  situation  worse.  (See 
5.1.2.)  [Source:  AHCI,  1998] 

D  5.15.3  Range  of  control  options.  Automated  systems  should 
provide  the  user  with  an  appropriate  range  of  control  options 
that  are  flexible  enough  to  accommodate  the  full  range  of 
operating  conditions  for  which  it  was  certified.  [Source:  AHCI, 
1998;  Parasuraman  &  Riley,  1997;  Sarter  &  Woods,  1995] 

Discussion.  Highly  flexible  automated  systems  can  be 
useful  when  the  user  knows  how  to  implement  the 
various  options  across  a  wide  spectrum  of  operational 
situations.  However,  the  multiple  options  that  are 
associated  with  highly  flexible  systems  also  require 
additional  cognitive  resources  in  order  for  the  user  to 
remember  which  mode  is  active.  [Source:  Woods,  1996] 
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■  5.15.4  Immediate  feedback.  To  promote  successful  situation 
awareness  of  the  automated  system,  the  user  shall  be  given 
immediate  feedback  to  command  and  control  orders.  [Source: 
Morris  &  Zee,  1988] 

D  5.15.5  System  flexibility.  Control  automation  should  be  flexible 
enough  to  allow  for  different  user  styles  and  responses  without 
imposing  new  tasks  on  users  or  affecting  automation 
performance.  [Source:  Wiener  &  Curry,  1980;  Woods,  1996] 

■  5.15.6  Override  and  backup  alternatives.  Override  and  backup 
control  alternatives  shall  be  available  for  automation  controls 
that  are  critical  to  the  integrity  of  the  system  or  when  lives 
depend  on  the  system.  [Source:  Billings,  1991] 

■  5.15.7  Backup  information.  Information  for  backup  or  override 
capability  shall  be  readily  accessible.  [Source:  Billings,  1991] 

D  5.15.8  Overriding  out-of-tolerance  conditions.  When  a  user 
might  need  to  operate  in  out-of-tolerance  conditions,  then  a 
deliberate  overriding  action  should  be  possible.  [Source:  Billings, 
1991] 


Discussion.  There  may  be  cases,  particularly  in  an 
emergency  situation,  when  the  user  needs  to  operate  in 
out-of-tolerance  conditions.  [Source:  Billings,  1996] 
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GLOSSARY 

Automation  -  a  device  or  system  that  independently  carries  out  a  task  that  was  formerly  carried 
out  by  a  human. 

Control  automation  -  when  an  automated  system  executes  actions  or  control  tasks  with  some 
level  of  autonomy. 

Decision  aids  -  (sometimes  referred  to  as  decision  support  systems)  automated  systems  that 
provide  support  to  human  decision-making  processes  either  unsolicited  or  by  user  request. 
Decision  aids  can  narrow  the  decision  alternatives  to  a  few  or  suggest  a  preferred  decision  based 
on  available  data. 

Direct  manipulation  -  when  the  user  controls  the  interaction  with  the  computer  by  acting 
directly  on  objects  on  the  display  screen.  An  object  may  be  an  icon,  menu  option,  symbol,  button, 
or  dialog  box.  An  example  of  direct  manipulation  is  a  GUI. 

Mental  model  -  an  individual’s  understanding  of  the  processes  underlying  system  operation. 
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